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(54) Motion compensated filtering or interpolation of interlaced television signals 

(57) A video processing method and system for motion compensated filtering of interlaced television signals 
comprising a motion estimation device for producing motion vectors corresponding to each output sampling 
site, a vector processor generating a set of input coordinates for each output sampling site depending on its 
corresponding motion vector, a data store in memory for storing input pixel values and providing a variable 
delay, a plurality of multipliers and corresponding coefficient stores, and an adder to sum the partial result 
from each multiplier, the multipliers multiply an input pixel value, selected from the data store depending on a 
part of the input coordinate, by a coefficient selected from the coefficient store depending on another part of 
the input coordinate, the filter aperture (set of filter coefficients) is selected depending on the vertical velocity 
component of the motion vector supplied by the motion estimation device. 
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MOTION COMPENSATED FILTERING 

This invention relates to improved motion compensated 
filtering or interpolation of interlaced signals and in 
particular video sequences such as television pictures, with 
application in all areas where interlaced video sequences are 
processed using motion vectors, including (but not limited 
to) : 

i. video compression 

ii . video to film conversion 

ill. motion compensated colour decoding 

iv. video motion estimation 

v. television standards conversion 
vi . slow motion display 

In this application the term video sequence refers to an 
image sequence sampled in an interlaced fashion. An image may- 
be a mu 1 t i - dimens ional representation of some property 
(typically, but not necessarily brightness) of a scene, thus a 
moving image is a temporal and spatial representation of a 
property of a scene and an image sequence is a moving image 
sampled at specific instants. The instant at which a point in 
the moving image is sampled may depend on the spatial location 
of that point . A pixel is a sample of an image having a value 
corresponding to some property of the scene which the image 
represents and corresponding to a specific image co-ordinate. 
The image co-ordinate, a specification of the location of a 
point in an image, need not correspond to a point at which a 
sampled image has been sampled. Hence image co-ordinates 
(input and output) are not necessarily quantised. A sampling 
site is the image co-ordinate of a pixel in a sampled image. 

Conventional motion compensated filtering or 
interpolation is described in detail in references 2, 3, 4, 
5,10,12,13,14,16,17,18 & 20 listed in appendix 1. This process 
will henceforth be referred to as MCF (or conventional MCF) 
which will be assumed to include the process of interpolation 
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as well as filtering. 

The hardware implementation of a conventional motion 
compensated filter /interpolator is described in references 3, 
4, 18 & 20. One simple implementation is as a cascade of 
filter/ interpolation cells as illustrated in figure 1. 

Figure l shows a conventional MCF filter cell having a 
vector processor which converts the motion vectors for each 
output sampling site in an output image into the input 
coordinates of an input image used in the filtering or 
interpolation operation. 

The vector processor receives the output sampling site, 
generated by counters as described in the references, and the 
associated motion vector from motion estimator hardware 
(described in the references) and searches through consecutive 
fields to find the correct input co-ordinate corresponding to 
the output sampling site concerned. 

The motion estimator is a device for estimating the 
motion vectors corresponding to a moving image. The estimated 
motion vectors can be sampled on either the input or output 
sampling lattices for an image processing system. For MCF 
input and output sampling lattices are (usually) identical. 
For motion compensated interpolation, the estimated motion 
vectors should be sampled on the output sampling lattice. 

The input coordinate depends on the output sampling site, 
the motion vector, and the location of the filter tap within 
the filter aperture. A variable delay supplies a multiplier 
with the correct input pixel value (from an input image to an 
image processing system) for the current output sampling site 
which is determined by the integer part of the current output 
sampling site. The multiplier multiplies the value of the 
pixel by a pr e - ca 1 cul a t ed coefficient held in a look up table 
stored in ROM. The coefficient is determined by the required 
filter characteristics and the sub-pixel, or fractional, part 
of the input coordinate motion vector supplied by the vector 
processor 



as shown in figure 1. 

A typical motion compensated transversal filter/ 
interpolator^ such as that used for television standards 
conversion, might have an aperture spanning four fields, 4 
field lines on each field and 2 pixels on each field line. 
Field line spacing is twice the picture line spacing which is 
the distance between vertically consecutive lines in a 
vertically sampled image . Vertically consecutive picture lines 
need not be sampled at the same temporal instant. For a 
(interlaced) video sequence, vertically consecutive picture 
lines will be on temporally adjacent fields. Such a filter 
would, therefore, require 32 interpolation/filter cells 
cascaded together. In one possible implementation the variable 
delay could be implemented using dual port RAM (random access 
memory) . By making the dual port RAM able to store 4 complete 
fields, any point in the filter aperture is accessible from 
the variable delay. A delay is implemented by providing offset 
read and write addresses as described in reference 3, page 
179. The specific delay implemented for a given output pixel 
depends on the integer part of the motion vector for that 
pixel. The multiplier coefficients used are stored, as a 
lookup table, in ROM (read only memory) . The specific 
coefficient to be used is selected from the complete filter 
aperture by the fractional part of the motion vector. In this 
example each coefficient lookup table would contain l/32nd 
part of the complete f i 1 t e r / in t e r po 1 a t i on aperture. In other 
implementations the coefficient lookup tables might contain 
the complete aperture (set of coefficients) (reference 3) . For 
convenience the coefficient lookup table could be combined 
with the multiplier in a look-up table (implemented in ROM) as 
shown in reference 4 . 

The size of the f i 1 te r / int e rpolat ion coefficient lookup 
table, for a conventional motion compensated filter, is 
determined by the size of the filter aperture and the 
precision of motion vectors supplied to the filter. Consider, 




or horizontal pixel spacing per field. The size of the 
coefficient lookup tables is calculated as shown below. The 
number of spatial phases corresponds to the spatial precision 
of the motion vectors. The number of temporal phases 
determines the temporal precision of the interpolation (see 
references 3, 6 & 7), typically there are 16 temporal phases 
for a commercial standards converter. 

Horizontally 2 taps 16 phases - 5 bits 

Vertically 4 field lines and 16 phases 

= 8 picture lines and 16 phases = 7 bits 

Temporally 4 fields and 16 phases = 6 bits 

TOTAL = 1^ ^^^^ 

In this example the total coefficient lookup table size 
has an address space of 18 bits. The lookup table is 
partitioned into 32 separate smaller lookup tables, one for 
each interpolation cell, hence the size of each lookup table 
is only 13 bits. In other implementations the coefficient 
lookup table might have to contain the complete aperture. 
The difference between a motion compensated filter and 
interpolator is simply that a filter only has a single 
temporal phase whereas an interpolator has multiple temporal 
phases. Typically a filter will generate an output temporally 
co-sited with the input fields. Multiple phase, or polyphase 
filters are described in detail in reference B. 

It is highly desirable to prevent aliasing in video 
processing as it is visually very disturbing. In conventional 
(spatio-temporal) MCF a single interpolation aperture (set of 
filter coefficients) is used. This process is unable to remove 
undesirable signal aliases due to the interlaced nature of the 
video signal. These unwanted aliases, present in the output of 
a conventional MCF, can significantly degrade the performance 
of a video processing system using MCF. The reason for this 
unwanted aliasing is that a single interpolation aperture is 
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used for all motion speeds. Furthermore, the filter aperture 
is generally designed considering only the magnitude of the 
(spatio-temporal) signal spectrum ignoring phase. 

Accordingly the invention provides a filter or 
Interpolation device for processing video sequences comprising 
a plurality of filter cells having between them a filter 
aperture which varies depending on the vertical velocity 
components of motion vectors. 

The invention also provides a system for motion 
compensated filtering of interlaced television signals 
comprising a motion estimation device for producing motion 
vectors corresponding to each output sampling site, a vector 
processor generating a set of input coordinates for each 
output sampling site depending on its corresponding motion 
vector, a data store in memory for storing input pixel values 
and providing a variable delay, a plurality of multipliers and 
corresponding coefficient stores, the multipliers multiply an 
input pixel value selected from the data store depending on a 
part of the input coordinate by a coefficient selected from 
the coefficient store depending on another part of the input 
coordinate, the filter aperture {set of filter coefficients) 
is selected depending on the vertical velocity component of 
the motion vector supplied by the motion estimation device, 
and an adder to sum the partial result from each multiplier. 

The filter or interpolator may comprise a coefficient 
look-up table stored in memory, wherein the total number of 
different coefficient sets available is given by 2^*^, where p 
is the vertical vector precision in bits. The vertical vectors 
being measured in units of input picture lines/ field period. 



aperture according to the phases of the signal and alias 
components of the spatial spectra of the input fields. The 
method is achieved by using a filter aperture which depends on 
the vertical velocity associated with each output sampling 
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site. Differen: filter apertures are used for different 
velocities and the filter aperture used can vary between 
adjacent output sampling sites. 

The method may comprise assigning a motion vector to each 
output sampling site, producing a set of input coordinates for 
each output sampling site, selecting a filter aperture (set of 
filter coefficients) from multiple pre - calculated filter 
apertures depending on the vertical part of each motion 
vector, selecting a coefficient from the filter aperture 
depending on the fractional part of each input coordinate, 
selecting an input pixel value depending on the integer part 
of each input coordinate (where input co - ordina.tes are 
measured in input picture lines per field period) , and 
multiplying each input pixel value by the correspondingly 
selected coefficient to produce a partial result for each 
output sampling site and summing the partial results. 

The number of filter apertures required is dependent on 
the precision of the vertical component of the motion vector. 

Improved filtering is achieved by designing the filter's 
frequency response, based on the phase and magnitudes of the 
signal and alias components in the spatial spectra of input 
fields, in order to cancel the unwanted alias components 
leaving only the desired signal components. 

This invention, by designing filter apertures as 
described above and detailed below and adapting the 
filter /interpolation aperture to the motion vector, enables a 
significant reduction of unwanted aliasing at the output of a 
motion compensa::ed f 1 1 1 e r / i n t e rpol a t or . By considering the 
phase and magnitude of signal and alias components of 
consecutive spatial spectra it is possible, as described 
below, to design a filter aperture that substantially cancels 
aliasing which would otherwise be present in the output of a 
conventional MCF . To achieve this a different aperture is 
required for each different vertical motion speed. In 
practical terms this is achieved by selecting a different 
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f i 1 t er / in t erpo 1 a t ion aperture depending on vertical motion 
speed. Quantisation of the vertical motion vectors supplied to 
the motion compensated filter ensures that only a finite 
number of different filter apertures are required. This 
invention might, concisely, be described as vertical velocity 
adaptive motion compensation filtering (VAMCF) . 

The invention will now be described by way of example 
with reference to the accompanying drawings, in which: 
Figure 1 is a block diagram of a motion compensation 
filter/interpolation cell according to a prior art 
implementation of motion compensated filtering; 
Figure 2 is a block diagram of a VAMCF/ interpolation cell 
according to a first embodiment; 

Figure 3 is a block diagram of an alternative VAMCF/ 
interpolation cell according to another embodiment; 
Figure 4 is a block diagram of a method of interlace to 
progressive conversion; 

Figure 5 is a block diagram of a composite video decoder; and 
Figure 6 is a schematic of field timing for calculating field 
difference signals; 

In the method of the invention one of many different 
interpolation apertures is used for filtering depending on the 
vertical velocity component of a motion vector. With 
appropriate filter design (as described in appendix 2) the 
aliasing present in the interlaced input can be cancelled to 
produce an output substantially free from unwanted alias. In 
order to do this different interpolation apertures are 
required for different vertical motion speeds. The number of 
different filter apertures required depends on the vertical 
vector precision- Assuming the vertical motion speed is 
measured in picture lines/field period then the number of 
different coefficient sets required is given by: Sets = 2^*^ 
(where P is the vertical vector precision in bits) , 

So for vertical vectors measured to a precision of 1/16 
picture line/field period 2^*^ = 32 different coefficient sets 
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are required. The P+1 (5 in this example) bits required to 
address the coefficient sets are the fractional vertical 
vector bits plus the least significant integer vertical vector 
bit. VAMCF can therefore be implemented according to Figure 2. 

The filter cell shown in Figure 2 differs from that shown 
in figure 1 (described above) in that the coefficient table 
contains multiple filter apertures (sets of filter 
coefficients) and is, therefore, considerably larger and the 
filter aperture is selected by the vertical velocity component 
of the motion vector. 

TO implement, in this example, VAMCF rather than MCF, the 
size of the coefficient lookup table would increase from a 13 
bit address (see above) to an 18 bit address. In an 
implementation which requires all the coefficients (from all 
apertures) to be stored in a single lookup table the address 
space would increase from 18 to 23 bits. In either case this 
is a significant increase in the size of lookup table 
r equi red. 

For VAMCF there may be an advantage to combining the 
vertical parts of the vector processing and the coefficient 
lookup table. This may reduce the size of lookup table 
required because the vertical part of the coefficient address 
and the aperture (coefficient set) address contain redundant 
information. Given the vertical motion speed and the temporal 
coefficient address, both the vertical coefficient address and 
the aperture (coefficient set) address can be found. This 
results in the implementation of figure 3. 

The best implementation (figure 2 or figure 3) depends on 
the vertical aperture size and the maximum vertical velocity. 
The size of the look up table (implemented in ROM) for figure 
3, assuming a maximum vertical vector of +/- 32 pixels (ie. 
picture lines) /field period , is given by (in this example) ; 
Horizontal 2 taps 16 phases = 5 bits 

Temporal 4 fields 16 phases = 7 bits 

Vertical +-32 picture 1 i n e s / f i e 1 d , pr e c i s i on l/16=9bits 



TOTAL ~ 21 bit-S 

which is 2 bits less than the implementation of figure 2. 

The filter cells shown in Figures 2 and 3 contain 
modifications to the conventional motion compensated filter, 
shown in figure 1, which result in motion compensated filters/ 
interpolators that adapt to vertical motion speed to cancel 
aliasing. The use of different interpolation apertures for 
different vertical velocities allows aliasing inherent in 
interlaced television signals to be cancelled, thereby 
preventing degradation of the output of motion compensated 
filtering/interpolation systems . 

Motion compensated filtering requires a minimum filter 
aperture spanning at least 2 fields but better results can be 
achieved, as indicated in appendix 2, by using more than two 
fields - 
Example 1 

Interlace to progressive conversion for video compression. 

Many video compression systems (e.g. iJPEG , MPEG, sub-band 
coding, wavelet transform schemes) perform better with 
progressive rather than interlaced pictures. Many video 
sources, however, produce interlaced rather than progressively 
scanned video. Hence to get the best out of compression 
systems requires the conversion of interlaced video to a 
progressive format. Existing interlace to progressive 
converters, be they a fixed filter, motion adaptive, or motion 
compensated, are unable to perform this conversion without 
leaving considerable aliasing in the progressive output. This 
aliasing can significantly degrade the performance of the 
video compression system. 

Using the improved motion compensated filtering of this 
invention, the quality of the interlace to progressive 
conversion can be significantly improved by reducing the 
unwanted aliasing in the progressive signal. This, in turn, 
allows improved quality or a reduced bit rate from the 
compression system. An interlace to progressive converter is, 
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conceptually, a filter as illustrated in figure 4. 

The interlaced input is first padded with zeros (as is 
described in many texts on digital signal processing) to 
create a highly aliased progressive signal. The zero padded 
signal is then filtered to remove as much aliasing as 
possible. The filter can be a fixed linear filter, motion 
adaptive filter, non-linear (for example median) filter, or a 
conventional motion compensated filter. If an alias cancelling 
motion compensated filter is used (that is a VAMCF according 
to the invention) a substantially alias free progressive 
output can be produced. 

The alias cancelling motion compensated filter (VAMCF) 
used for interlace to progressive conversion requires an 
aperture spanning at least two fields. However, this provides 
no immunity from signal noise or inaccuracies in the motion 
vectors used. A two field filter also has the disadvantage of 
either a non-zero group delay (producing progressive output 
frames half-way between input fields) which is inconvenient, 
or an asymmetric aperture which may produce inferior results. 

A better implementation of alias cancelling motion 
compensated filtering (VAMCF) for interlace to progressive 
conversion would use a 3 (or more) f ield' "aperture . This would 
provide a symmetric aperture with zero group delay (output 
frames co-timed with input fields) and some immunity from 
input signal noise and motion vector errors. Although 3 fields 
provide more information than is necessarily required for 
interlace to progressive conversion (mathematically an 
ove r - de t ermine d problem) , the output frames can be determined 
as at least mean square ( LMS ) error approximation to all 3 
input fields. This LMS approximation (described in appendix 2) 
provides the robust characteristics described above. In the 
computation of the filter characteristic to give the "best" 
(LMS) output frame, it is not necessary to weight the errors 
from all 3 fields equally. A better solution would probably be 
to weight the errors from the central field of the aperture 
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twice as heavily as the two flanking fields. This weighted 
LMS approach to filter design is described in appendix 2. 

Obviously it is possible to implement a VAMCF with more 
than three fields in its aperture. As the number of input 
fields increases, immunity to input noise and motion vector 
errors increases, but so does hardware complexity and the 
assumption of constant motion (implicit in motion 
compensation) across many fields is more likely to be 
violated. In practice a good compromise is probably to use a 3 
or 5 field filter. 
Example 2 

Video to Film Conversion. 

It is sometimes desirable to be able to convert a video 
signal for recording onto film ( tel e - re cording ) . This might be 
done for archive purposes or for inserting a video sequence 

(perhaps with special effects) into a motion picture film. 

A 50 f i e 1 ds / se cond video sequence could be recorded onto 
25 frame/second film simply by recording 2 fields on each 
frame. Similarly 60 f i e 1 ds / s econd video could be recorded on 
to 30 frame/second film. However, recovering the original two 
fields from the recorded film frame would be very difficult 
because of registration problems and spatial distortions in 
both the optical recording and playback (telecine) operations. 
If the separate fields were confused in the replayed signal 

(as is likely) highly disturbing impairments would results in 
moving scenes. Even if the fields could be recovered correctly 
from the recorded film, motion rendition of paired fields 
would probably be unacceptable when the film was displayed 
using an optical projector. 

To avoid the problems of recording paired fields it is 
necessary to convert interlaced signals to a progressive 
format. Single progressive frames can then be recorded on each 
film frame . This technique avoids the problems associated with 
recording paired fields described above. The interlace to 
progressive conversion process for t el e - r e c o r di ng can be 
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performed in exactly the same way as for video compression 
(example 1 above) using VAMCF to provide improved quality. The 
only difference with te le - recording is that, for example given 
a 50Hz interlaced source, only a 25Hz (rather than 50 Hz) 
progressive output is required for tele - recording . 

It is probably also worth noting that video graphic post 
production systems often perform best with a progressively 
scanned input. This progressive input can be also obtained 
using a VAMCF, as described above, to give improved quality. 
Example 3 

Motion compensated colour decoding 

Many techniques are described in the literature for 
decoding composite colour television signals (PAL and NTSC) . 
One of these variations is shown in figure 5. 

This configuration contains two (different) baseband 
chrominance pass filters. The chrominance pass filters can be 
1 dimensional (horizontal), 2 dimensional (spatial) or 3 
dimensional (spatio-temporal) filters. Mul t i - d imens ional 
linear filters are usually referred to as 'comb filters' in 
the context of colour decoding. 

The comb filters in a colour decoding circuit can be 
replaced by conventional motion compensated spatio-temporal 
(comb) filters. This would produce a motion compensated colour 
decoder. This might provide improved performance over a 
non-motion compensated comb filter decoder. Unfortunately the 
output of the motion compensated comb filters would still 
include alias components which would impair 

chrominance/luminance separation. If instead a VAMCF was used 
to replace the comb filters in figure 5, aliasing at the 
output of filters would be reduced, resulting in improved 
luminance/chrominance separation. The filter coefficients for 
such a filter would be designed to take account of both the 
magnitude and phase of the (demodulated) composite signal, but 
a VAMCF would sti: be required to avoid aliasing in the 
filter output. 
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Example 4 

Motion estimation using interlaced video signals. 

In many techniques for video motion estimation 

(references 1, 2, 2, 9, 11, 13, 4,19 & 21), it is necessary to 
compare displaced versions of input pictures to test the 
validity of a possible motion vector. Typically, the current 
input picture is subtracted from a displaced version of the 
preceding or subsequent picture of a sequence. Unfortunately, 
if the input is an interlaced sequence, the displaced field 
difference signal contains significant energy due to aliasing 
implicit in the interlaced signal. This alias energy cannot be 
removed by intrafield filtering prior to motion estimation. 
Aliased energy can confuse a motion estimator, causing it to 
generate erroneous motion vectors with deleterious effects to 
down stream processing using the motion vectors. 

By using alias cancelling motion compensated filtering 

(VAMCF) , substantially alias free displaced field difference 
signals can be generated for use in motion estimation. This 
affords improved motion estimation by reducing the chance of 
the motion estimator being confused by aliasing. In order to 
generate alias cancelled displaced field difference signals it 
is necessary to use at least three fields' rather than the 
usual 2 . 

A conventional forward (or backward) field difference 
signal is generated by a conventional motion compensated 
filter with a 2 field aperture. The aperture used is a field 
difference, that is a temporal high pass filter, and may 
include spatial ( intra - f ie Id ) interpolation. The motion vector 
used is the vector under test. This process could, 
alternatively, be considered as first calculating the mean of 
the two fields and then subtracting the mean field from each 
of the two input fields. This gives 2 (identical} error 
signals. If the fields were progressively scanned, rather than 
being interlaced, the mean field would be a least mean square 
(LMS) best fit to the two input fields. 
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To generate an alias cancelled field difference, 
conceptually one first calculates a LMS estimate of a 
progressive frame from 3 or more input fields . Next a 
displaced field difference signal can be calculated from that 
progressive frame, for each of the input fields in the 
aperture . 

Typically one might use 3 fields (i.e. fields 0,1 and 2 
illustrated in figure 6), although more fields (but not less) 
are possible, with a 3 field aperture, 3 distinct field 
difference signals can be calculated rather than the single 
unique) displaced field difference of the conventional 
approach. Any one of these field differences could be used in 
place of the conventional displaced field difference. 
Alternatively the multiple field differences, provided by the 
alias cancelling approach, could be combined together (for 
example by taking sum of squares) to provide a more useful 
measure of the validity of a vector than could be obtained 
using the single field difference alone. 

The calculation of alias cancelled field differences can 
be implemented using VAMCF as described earlier. To calculate 
each field difference requires a separate (spatio-temporal) 
filter. The filter design technique for fhis application is 
outlined in the appendix 2. 
Example 5 

Television standards conversion 

Television standards conversion is the process of 
converting between different television standards, for example 
between 62 5 line 50Hz PAL and 525 line 60Hz NTSC . Assuming 
colour decoding and re-encoding are performed separately, 
standards conversion is a 3 dimensional (spatio- temporal) 
interpolation problem. The theory and techniques of standards 
conversion are described in detail in references 3, 6 & 7. 
Motion compensation has been shown to provide improved quality 
compared to non-motion compensated standards conversion. 
Conventional MCF, however, still leaves some aliasing in the 
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output pictures with consequent degradation to picture 
quality- This degradation would be particularly evident for 
slowly moving, high resolution images such as computer 
graphics. These sort of pictures could be processed admirably 
using motion compensation were it not for this residual 
al ias ing . 

VAMCF can be directly applied to television standards 
conversion. In this application removing aliasing results in a 
direct improvement in picture quality. Standards conversion 
includes a temporal interpolation process. It is necessary to 
generate output fields at different times from the input 
fields. This is in contrast to the filter examples presented 
above where output fields are co-timed with the input fields. 
The design of filter coefficients for temporal interpolation 
is more complex than for filtering and is outlined in appendix 
2 . 

Example 6 

Slow motion displays 

It is often desirable to replay a video signal more 
slowly than the original action. A common example of this is 
'action replays' in television sports coverage. One way to 
achieve this is to simply repeat fields 6"r frames as necessary 
to provide a sufficient picture rate. This technique is 
commonly used in video recorders to achieve slow motion 
replay. Unfortunately it results in an unnatural and 
disturbing 'jerky' motion rendition. Motion compensation can 
be used to provide improved motion rendition as described in 
reference 18. Conventional motion compensation, however, 
leaves residual aliasing in the output signal. In a slow 
motion replay system, aliasing from vertical detail (for 
example the top edge of a wall) can result in wobbling detail 
on an otherwise stationary image. This can be visually very 
di s t urbing . 

VAMCF can be used for motion compensated slow motion 
replay in the same way as for television standards conversion. 
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The theory is essentially identical but with a greater number 
of output fields per input field. In this case, the removal of 
unwanted aliasing is even more important than for standards 
conversion because the aliasing is more visible on the slowed 
down s ignal . 
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Appendix 2 : 

For simplicity this appendix only considers the vertical 
and temporal dimensions. Extension to include the horizontal 
dimension is straight forward for those skilled in the art. 

Consider an object, denoted h, moving with a velocity, v, 
to produce a moving image, denoted g. 

g ( y , t ) =h ( y- V . t ) Equation 1 

In a television system the continuous, spatio-temporal, 
image g is sampled on an interlaced sampling lattice. The 
spatial position of the sampling grid alternates between odd 
and even positions on consecutive fields. Two sampled signals, 
S, are possible for each field position depending on whether 
the image, g, is sampled with an even or odd field lattice. 
This results in two distinct possible si g_n als for each field 
denoted Se and So Taking the Fourier transform of an even or 
odd sampled signal gives the following result. 

Equation 2 

In these equations n represents vertical frequency, 
script F (subscript y) represents a Fourier transform (with 
respect to y only) , H is the Fourier transform (complex 
spectrum) of the object, h, and Y is picture line 
spacing (field line spacing is 2Y) . The equations consider 
first order aliasing only and are valid for positive 
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frequencies only ( simi lar equations apply for negative 
frequencies) . 

Consider the example of interlace to progressive 
conversion using 2 input fields. Assume the two input fields 
are at t = 0 and t = T {T is the field period) and that the 
first field is even and the second field odd. Taking the 
Fourier transforms of the two fields yields the following 
matrix equation. 

rsx»./=o)] r > ^ ( ( i\ J/^"n 



or 
S = P.H 



Equation 3 



Where S is a vector of the spectra of sampled fields, P 
is a matrix of phase factors (similar to a discrete Fourier 
transform matrix) and H is a vector of the object spectrum and 
alias spectrum. 

In calculating the progressive frame from the interlaced 
fields we are trying to find the unsampled image at t = 0 from 
the sampled (interlaced) fields. The unsampled image we 
require is h(y) . By solving equation 3 we. can find H(n) (and 
also H(n - 1/2Y) , the aliasing, which we do not require) and 
then h(y) by an inverse Fourier transform. 

-1 



H = P-\S 



or 



1 + 



1 -expQljznvt) 



r.(n,r-0)] 



Equation 4 



Considering the expression for H(n) only we have; 
H(n) = 



expf y^^^j 



Equation 5 
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inverse Fourier transforming equation 5 gives; 

h(y)= filter^,(y,v)*S^(y,t = o ) + f il ter ^ ( y , v) ♦ (y , t = T) 

« f ilter (y, t , v) ♦S (y. t) Equation 6 

Where * represents the convolution operation. 

Equation 6 shows that we can find the progressive image, 
h(y), that we require by filtering each of the interlaced 
fields separately using intrafield filters and then summing 
the results. Summing two intrafield filters on consecutive 
fields can be considered as using a single spatio-temporal 
filter. The intra-field filters used are given by the inverse 
Fourier transforms of the coefficients of and in 

equation 5. Note that the filters in equation 6 are functions 
of V, the vertical motion speed. 

The filters of equation 6 represent an idealised 
situation. The filters have infinite extent and therefore 
cannot be realised in practice. However, the coefficients of 

S and S in equation 5 represent the frequency domain 
e o 

specifications for the ideal interlace to progressive 
conversion filters. There are many techniques in the 
literature (see for example reference 15) for designing 
practical filters from an ideal frequency domain 
specification. 

The performance of an interlace to progressive converter 
can be improved by using more than a two field aperture as 
discussed in the main text. Consider a 3 field aperture with 
fields at t = -T, 0 & +T . Equation 3 then becomes; 



1 } \ ^ 



or 
S = P-H 



Equation 7 



in this case the phase matrix is rectangular, not square, 
and it is no longer possible to directly invert equation 7 to 
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find H(n) . It is, however, possible to find a least mean 
square approximation to H(n) by solving the 'normal* equations 
given by; 

P**.S « {P**.P)H Equation 8 

Here the symbol, ^, represents the Hermitian conjugate 
(complex conjugate of the transpose) of the matrix. The 
above H indicates that H is a least mean square approximation. 
Solving for H gives; 

H = (P**. P) P**. S Equation 9 

The LMS solution for H again gives a frequency domain 
specification for the aliasing cancelling filters, practical 
versions of which can then be calculated using many published 
technique s . 

With a 3 field aperture, for interlace to progressive 
conversion, it may be appropriate to give greater prominence 
to the centre field in the aperture. For example it might be 
appropriate to weight the 2 outer odd fields half as much as 
the central even field. This can be done by multiplying 
equation 7 by an appropriate weighting matrix for example; 
W-S := W.P.H Equation 10 

^1/4 0 0 
0 1/2 0 
iO 0 1/4 ; 

Equation 10 can then be solved to find a weighted least 
mean square solution for H; 

fi= ( (P*^. . P) "-^ . (P^. W^) ) S Equation 11 

Again practical filters can be designed from the 
frequency domain specifications embodied in equation 11. 

The design of interpolation filters for standards 
conversion is similar to the design of interlace to 
progressive conversion filters. The process of standards 
conversion could be looked on as calculating a sequence of 
progressive frames from an interlaced input. Typically an 
interlaced output is required from a standards converter. This 
is easily achieved, conceptually, by simply spatially 



where ; W = 
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subsampling the progressive fields appropriately to give an 
interlaced signal. In practice this means that only half the 
number of output points need be calculated with a 
corresponding saving in the hardware required. 

For standards conversion, in contrast to interlace to 
progressive conversion, output fields are required at 
different time instants to the input fields. The effect of 
this is to change the phase terms in the phase matrix P. The 
elements of the phase matrix are still calculated using 
equation 2 as before. For standards conversion the values of t 
used are no longer integer multiples of a field period. Other 
than this small change the equations above still apply. Hence 
filter design for standards conversion is, in essence, very 
similar to that for interlace to progressive conversion. 

In motion estimation it is often required to test the 
validity of a possible mc ion vector by comparing consecutive 
fields. Unfortunately this comparison is, conventionally, 
complicated by the interlaced nature of video signals. 
Improved results are possible by correctly allowing for the 
interlaced nature of the video signal. 

Conceptually the comparison of cons e"c utive fields, 
allowing for interlace, is a two stage process. First a 3 {or 
more) field aperture is used to calculate a least mean square 
(LMS) estimate of a progressive frame assuming the test motion 
vector is correct. Then the input fields are compared to the 
expected fields given that the progressive frame and motion 
vector are correct and allowing for interlace. The least mean 
square estimate of (the complex spectrum of) the progressive 
frame is given by equation 12. 

H= (P^. P) "^.P*^. S Equation 12 

If the estimated spectrum, H, is correct then the input 
fields would be given by; 

S = P.H Equation 13 

The differences between the actual fields, S, and 
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thees timat ed fields, S, is given by; 

S-S= (I-P. (P**.P) -^•P*^) S Equation 14 

Equation 14 gives a frequency domain specification, for 
an alias cancelled field difference signal, for each field in 
the aperture. Practical filters can then be designed from the 
frequency domain specifications using published techniques. 



CLAIMS 

1 A filter or interpolation device for 
processing video sequences and comprising a plurality of 
filter cells having between them a filter aperture which 
varies depending on the vertical velocity components of 
motion vectors. 

2 A filter or interpolation device as claimed 
in claim 1, wherein each filter cell comprises a 
multiplier and a corresponding coefficient store. 

3 A video processing system for motion 
compensated filtering of interlaced television signals 
comprising a motion estimation device for producing 
motion vectors corresponding to each output sampling 
site, a vector processor generating a set of input 
coordinates for each output sampling site depending on 
its corresponding motion vector, a data store in memory 
for storing input pixel values and providing a variable 
delay, a plurality of multipliers and corresponding 
coefficient stores, and an adder to sum the partial 
result from each multiplier, the multipliers multiply an 
input pixel value selected from the data store depending 
on a part of the input coordinate by a coefficient 
selected from the coefficient store depending on another 
part of the input coordinate, the filter aperture (set of 
filter coefficients) is selected depending on the 
vertical velocity component of the motion vector supplied 
by the motion estimation device. 

4 A filter device as claimed in claim 2 or a 

video processing system as claimed in claim 3, comprising 
a coefficient look-up table stored in memory, wherein the 
total number of different coefficient sets available is 
given by 2^*^, where p is the vertical vector precision 
in bi ts . 



5. A video processing method for processing 
interlaced video signals which, adapts the 

filter /interpolator aperture according to the phases of 
the signal and alias components of the spatial spectra of 
the input fields. 

6. A video processing method for processing 
interlaced video signals as claimed in claim 5 , wherein 
the filter aperture depends on the vertical velocity 
associated with each output sampling site, such that 
different filter Apertures are used for different 

vel oci t ies . 

7. A video processing method for processing 
interlaced video signals as claimed in claim 5 or 6, 
comprising assigning a motion vector to each output 
sampling site, producing a set of input coordinates for 
each output sampling site, selecting a filter aperture 
(set of filter coefficients) from multiple pre - cal cul at ed 
filter apertures depending on the vertical part of each 
motion vector, selecting a coefficient from the filter 
aperture depending on the fractional part of each input 
coordinate, selecting an input pixel value depending on 
the integer part of each input coordinate (where input 
co-ordinates are measured in input picture lines per 
field period) , and multiplying each input pixel value by 
the correspondingly selected coefficient to produce a 
partial result for each output sampling site and summing 
the partial results. 

8. A video processing method for processing 
interlaced video signals as claimed in claim 5, 6 or 7, 
wherein the number of pre calculated filter apertures is 
dependent on the precision of the vertical component of 
the motion vector. 



Amendments to the claims have been filed as follows 

3^ A filter or interpolation device for motion 

compensated filtering or interpolation of interlaced 
video signals and comprising a plurality of filter taps 
implementing between them a filter aperture which varies 
depending on the vertical velocity components of the 
motion vectors used. 

2 A filter or interpolation device as claimed 
in claim 1, wherein each filter tap comprises a 
multiplier and a corresponding coefficient store. 

3 ^ A video processing system for motion 
compensated filtering of interlaced video signals 
comprising a motion estimation device for producing 
motion vectors corresponding to each output sampling 
site, a vector processor generating a set of input 
coordinates for each output sampling site depending on 
its corresponding motion vector, a data store in memory 
for storing input pixel values and providing a variable 
delay, a plurality of multipliers and corresponding 
coefficient stores, and an adder to sum the partial 
result from each multiplier, the multipliers multiply an 
input pixel value selected from the data store depending 
on a part of the input coordinate by a coefficient 
selected from the coefficient store depending on another 
part of the input coordinate, the filter aperture is 
selected depending on the vertical velocity component of 
the motion vector supplied by the motion estimation 
devi c e . 

4 A filter device as claimed in claim 1 or 2 o 
a video processing system as claimed in claim 3, 
comprising a coefficient look-up table stored in memory, 
wherein the total number of different coefficient sets 
available is given by 2^*^, where p is the vertical 
vector precision in bits. 
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5 . A video processing method for motion 

compensated filtering or interpolation of interlaced 
video signals comprising assigning a motion vector to 
each output sampling site, producing a set of input 
coordinates and selecting a filter aperture depending on 
the vertical velocity component of the motion vector 
associated with each sampling site, such that different 
filter apertures are used for different vertical 
ve loc i ties. 

6. A video processing method for processing 
interlaced video signals as claimed in claim 5, the 
filter aperture is selected from multiple pre - cal culated 
filter apertures depending on the vertical velocity 
component of each motion vector, and the method 
comprising selecting a coefficient from the filter 
aperture depending on the fractional part of each input 
coordinate, selecting an input pixel value depending on 
the integer part of each input coordinate, and 
multiplying each input pixel value by the correspondingly 
selected coefficient to produce a partial result for each 
output sampling site and summing the partial results. 

7. A video processing method for processing 
interlaced video signals as claimed in claim 5 or 6, 
wherein the number of precalculated filter apertures is 
dependent on the precision of the vertical component of 
the motion vector. 
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